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ABSTRACT 



The validity of the Dale-Chall Readability ratings 
for sixth-grade science textbooks when compared to an independent 
criterion of language difficulty expressed in cloze units was 
examined. The Canadian Lorge-Thorodike IQ Test and the Canadian Test 
of Basic Skills, Vocabulary and Comprehension subtests, vere 
administered to 366 sixth grader's. Mean IQ score was 102^. while mean 
scores for vocabulary and comprehension were 6.19 and 6.15, 
respectively. Passages from 12 science textbooks were subjected to 
the cloze technique and administered to the subjects. Findings 
revealed that (1) the 12 cloze passages yielded a reliability 
coefficient of .79; (2) cloze scores correlated from .55 to .85 with 

the intelligence measure; (3) cloze .i>cores correlated from .64 to .86 
with vocabulary, and from .52 to .85 with comprehension; (4) the 
cross-validation coefficient of .90 obtained by Dale and Chall and 
the cross-validation coefficient of -.29 obtained in this study are 
significantly different beyond the .01 level of confidence. .Major 
conclusions were (1) that the cloze tests are reliable measures of 
language difficulty and (2) that the Dale-Chall Beadability Formula 
is not a valid measure of sixth-grade science textbook materials when 
the cloze procedure is used as a criterion. References are included. 
(VJ) 
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CLOZE RE.^DABILI^rY VERSUS THE DALE-CHALL PORI-RJLA 

Research Report 
Thursday, April 22, 1971 
2;15>3:15 P.M. 

Heed for study 

Much contradictory evidence is found in the literature 
with regards to the measurement of roadability. Some of the 
most pertinent findings with regards to the roadability of 
elementary science textbook materials wlll~be examined in 
this paper. 

Since by generally accepted standards a readability for- 
mula is applicable only to material similar to the criterion on 
vihich it Vlas based (3) (10) » dt appears that the Dale-Chall for- 
mula , scores for elementary science textbook material should be 
used cautiously. Chall, in 1953, stated that no studies based 
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exclusively on the specialized subject matters of science or 
mathematics had been reported. A search of the literature 
to date revealed no validation studies done on elementary 
science materials. 

However, several studies reported findings based on the 
application of the Dale-Chall formula to materials which 
were not included in the original validation or subsequent 
validations. This would lead one to question the results of 
these’ studies. 

Brotm (2) questioned the appropriateness of the Dale 
list of 3000 familiar words as a vocabulary load factor in 
the Dale-Chall formula. He found that students at the seventh 
and eighth grade levels apparently comprehended a H.S.T.A. 
publication entitled Spacecraft which was rated at 11-12 
grade-level by the Dale-Chall formula. Furthermore, when 
the vocabulary from a I96I edition of a third-grade science 
textbook was accepted as familiar, it was found that it 
lowered the readability of the same sample from 11-12 grade- 
level to 9-10 grade-level. His conclusion vms that when the 
Dale-Chall formula was applied to science textbooks it 
seemed to place them higher than was warranted. 

Walker (3^) used the Dale-Chall foimula to evaluate 
thirty-nine commercially produced programmed textbooks for 
grades four to d. x and found that the Dale-Chall readability 
levels for these books were consistently higher than the 
grade levels assigned to them by the publisher. Sixty -seven 
percent of the books were rated above the grade level indi- 
cated by the publisher and three of the books had no samples 

2 ■' 
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at the intorroediate level. This type of finding may be 
questioned since the variability may be duo to either the 
nature of the formula or the material being evaluated. 

Although high relationships between the Pry, Lorge, Plesch, 
and Dale-Chall formulas have been reported by Pry (ij-), con- 
tradictory evidence is quoted by Hichaelis and Tyler (13) . 
Marshall (12) , furthermore, found no significant relation- 
ship between readability and comprehension when using the 
Plesch Reading Ease formula on high school physics books. 

Since most of the formulas are based on common or 
highly interrelated factors, and since the vocabulary factor 
accovints for a veiy substantial amount of the varitoice of 

I 

readability elements, this aspect of the foimulas must be 
examined for its contribution toward formula consistency or 
inconsistency. There is also some evidence to indicate that 
vocabulary difficulty is a better predictor at lower levels 
of difficulty and for poorer readers (^) . 

The Winnetka, Dale-Tyler, Gray-Leary, Lorge, Dale-Chall, 
bolch and Spacha forravilas all use basic word lists vjhich are 
dated since they were compiled in the 1920 ’s and 1930 ’s. 

Evidence gathered by Stone (15) and Jacobs (7.) indicates 
that vocabularies have changed. Kucera and Prancis (6) also 
found that of the Dolch 220 words of the 192.0 ’s, 82 words 
were not among their i960 compilation of the most frquently used 
220 words,, Recognizing the possibility that this. .change might 
affect the readability, Spache.has now adopted Stone’s revised 
list. The validity of the other formulas, however, could be 
questioned since their lists have not been revised. 
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Bormuth, in investigating a measure of \jord depth as a 
predictor of comprehension difficxilty in literature, science, 
and history materials, found it to predict differences in com- 
prehension of different subject matter written at the same 
readability level. This led him to suggest: 



In the past the assumption has been that such differences 
were caused by differences inherent in the content of the sub- 
ject matters themselves. Though this concept nas never been 
rigorously defined, it is given the labels of concept diffi- 
culty or idea density. (1) 



In other words he is questioning differences in readability com- 
monly attributed to the content areas. 

Chall, in her 1958 summary of readability research, stated 

I 

the need for cross-validation research into textbook materials. 
.She wrote; 



Moat of the existing validation studies have been on juvenile 
fiction. Since the formulas are used by educational publishers 
and textbook committees for evaluating textbooks for a particu- 
lar grade, we need to know how valid the predicted grade -place- 
ment indexes of the various formulas are whan compared to tested 
comprehension or to teacher or pupil judgement of diffic\ilty.(^).,'- 



In his 1963 investigation of readability research, Klare 
proposed the following areas for future research which are 
related to this study: The ure of new word-lists, specialized 

word-lists, specialized lists for specialized purposes, longer 
lists, criterion passage refinement, when of wh^re to uc<e spe- 
cific formulas, and separate norms for the readability of dif- 
ferent types of materials and audiences (8_) . 

It is apparent that the Dale -Chall formula has not been 
validated ori elementary science materials. The Dale List of 
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3000 familiar words may be questioned for relevance to science 
material. And the weighting of the vocabulary factor in the 
formula may need adjustment for certain types of materials or 
for certain levels of materials. 

Furthermore, differences in readability due to content ... 
materials have been questioned. Also some contradictory evi- 
dence about the relationship between readability and comprehen- 
sion has been presented. 

Finally, a need for readability research vjith respect to 
validation, use of word lists, and use on content material has 
been e3Q>ressod. 

' Further research, especially on the validity of using the 
Dale-Chall formula on elementary science textbook material, is 
therefore needed. 

The problem 

This study was designed to investigate the problem: How 

valid are the Dale-Chall Readability ratings for sixth grade 
science textbook materials when compared to an independent cri- 
terion of language difficulty expressed in cloze units'. 

Null hypothesis 

There is no significant difference between the correlation 
coefficient of .90 obtained in the original cross-validation of 
the Dale-Chall Readability Formula and the coefficient of corre- 
lation between the Dale-Chall readability ratings and the mean 
cloze score obtained by the subjects con^sleting the cloze test 
over randomly selected sixth grade science textbook passages. 



Definition of terms 
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1. CROSS - VALIDATIOH . A comparison of a predicted readability 
level to an independent criterion of readability (or lang- 
uage difficulty). 

2. CLOZE PROCEDURE (or CLOZE TECHNIQUE). A method of intercept- 
ing a message from a transmitter, mutilating its language 
patterns by deleting parts [in this case, every 5th word!] 
and so administering it to receivers that their attempts to 

- make the pattern whole again yields a niimbor of clozo units 
(the score) (l6) . 

3. INDEPENDENT CRITERION of LANGUAGE DIFFICULTY . A measure of 
the larigjxage difficulty of a passage expressed^|mean cloze 
vmits for the subjects having read a particular passage. 

4. READABILITY . A prediction of the ease or difficulty with 
which written material may be comprehended according to the 
spevcified criteria. 

5. SIXTH GRADE SCIENCE TEXTBOOK MTERIALS . Randomly selected 
samples of science materials from textbook series designated 
for fche sixth grade by the publisher and listed in Textbooks 

HI ih '- P.riht6l968" (17) . 

The popul at ion 

The s^ple \inder study consisted of 366 randomly selected 
sixth grade students enrolled in thirteen classrooms in eleven 
different schools in School District t 5^33» Chilliwack, British 
Columbia, Canada. The 366 students represented approximately o 
one-half of the sixth grade students in this district. 

The mean Canadian Lorge- Thorndike IQ (11_) for the 366 
students was 102 v/ith a standard deviation of l5. 
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Approximately 5l percent were girls and approximately lj.9 
percent were boys. 

The Canadian Test of Basic Skills (9) j subtest Vocabulary 
and Comprehension were administered sixty days before this 
study (in October) and resulted in a mean score of 6.19 
(SD=:1.27) and 6.15 (SD=1.26) respectively. 

Both the IQ test and the Basic Skills test were admini- 
stered as a regular part of the District testing program. 

Svunmary of findings 

Pour specific findings were obtained from the data in this 
study: 

1. The twelve cloze passages used in this study yielded a 

I 

mean reliability coefficient of .79 with a range of .61|. to .y9. 
These coefficients are comparable to some standardized group 
reading tests. 

2. The cloze scores obtained for each passage correlated 
from .55 to .85 with intelligence as measured by the Canadian 
Lor ge -Thorndike test. These coefficients are comparable to 
those found in studies by H8d*ner and Jenkinson (ill) . 

3. The cloze scores obtained for each passage correlated from 

^61i to ,86 with reading vocabulary, and from .52 to .85 with 
reading comprehension. These findings compare favorably with 
those cited by Hankin (111) . . 

ll. The cross-validation coefficient of .90 obtained by Dale 
and Chall and the cross-validation coefficient of -.29 obtained 
in this study are significantly different beyond the .01 level 
of confidence. 



Limitations 
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The findings and conclusions in this study must be inter- 
preted keeping in mind the relatively small nimiber of passages, 
the reliability and validity of the cloze tests, the specific 
samples of subjects, and the science' material used: 

1. An item sampling error was probably inherent in the cloze 
passages. 

2, Approximately i?5 percent of the passages were within the 
frustration levels of the examinees completing the passages. 

This factor could cause a leveling-off of the distribution of 
scores for the cloze passages at the upper end of difficulty. 
Conclusions 

I 

Subject .to the limitations of this study the following four 
conclusions were dram from the findings: 

1. The cloze tests are reliable measures of language diffi- 
culty when used for group testing. 

2. The cloze tests are valid measures of language diffi- 
culty as demonstrated by their concurrent validity i-riLth intel- 
ligence test scores. 

3. The cloze tests are valid measures of language difficulty 
as demonstrated by their congruent validity with rea^ng vocabu- 
lary and reading comprehension scores, 

Ij.. The Dale-Chall Readability Formula is not a valid measure 
of sixth-grade science textbook materials when the cloze proce- 
dure is used as a criterion. 
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Implications i 

Since the null hypothesis was rejected, and since the 

Dale-Chall Readability Formula was found not to be a valid mea- I 

sure of sixth-grade science matez’ial, the following implications’. 

should be considered: \ 

\ 

1. Previous research using the Dale-Chall Readability Formula i 

t 

I 

on elementary science textbook materials should be re-examined. | 

2. In order to validly measure science materials with the | 

i 

Dale-Chall Readability Formula the following alterations may be \ 

necessary: (a) Weighting the formula differently, (b) compil- ; 

j 

ing a specialized word list, (c) up-dating the word list, | 

(d) developing special noims, or (e) some combination of these. , 

3. For some purposes, such as matching a book to a reader, ■ 

the cloze procedure may be a more robust and more parsimonious ! 
measure of language difficulty than the Dale-Chall Readability ; 
Fomula. ! 

4. Reasearch using passages written specifically for a cer^ ^ i 
tain Dale-Chall readability' level should be interpreted cau-:^ j 
tiously since tills has, in effect, eliminated the readability | 

I 

measurement error. This could lead to different conclusions ; 
from those indicated when randomly selected passages are sub- ; 
jected to a readability formula. ! 

5. This study should be replicated at the same and at other * 
grade levels, with differing populations, and vri.th other randomly 

selected science textbook passages in order that the results ' 

* * 

could be more general! z able. ; 
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